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' We extend certain basic and general concepts of thermodynamics to discrete 

a ' Markov systems exchanging work and heat with reservoirs. In this framework we 

. show that the celebrated Clausius inequality can be generalized and becomes an 

^ i equality, significantly extending several recent results. We further show that achiev- 

Q I ing zero dissipation in a system implies that detailed balance obtains, and as a 

consequence there is zero power production. We obtain inequalities for power pro- 
duction under more general circumstances and show that near equilibrium obtaining 
maximum power production requires dissipation to be of the same order of magni- 
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00 ! Within the context of stochastic dynamics it turns out to be possible to make significant 

O ■ extensions of classical thermodynamics. For example, although the issue of power production 
is simply beyond thermodynamics (by definition), in the stochastic dynamical framework 
^ ', we are able to show that maximal power production entails a compulsory rate of dissipation 
^ I (essentially entropy production) of the same order when close to equilibrium. In other results 

to be reported below, we generalize the Clausius relation and develop circumstances under 
which it is an equality. 

We consider the stochastic dynamics of a discrete system s undergoing a discrete time 
process. The elementary time step r is taken as the time unit. The dynamics is defined by 
a stochastic matrix R = (Rxy), where Rxy = p{x, t + T\y, t) is the transition probability from 
1/ to X in time r. 
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Let 7 = (?/o, Ml, ^2, mat-i, Xq) be an step path from i/q to Xq. The weight of this path is 
The conditional average of a function F{'y,t), given the initial and final states yo and Xq, is 

where I7I is the number of steps in the trajectory 7. For each elementary transition y ^ x, 
such that Ryx and R^y differ from 0, we suppose that the following relation holds 

^ = eMSS^^y (3) 

where {6S^°^)xy is the total entropy variation of the system of interest, s, and of the other 
systems that are implied in the transition. Relation ([3]), which can be deduced from micro- 
scopic detailed balance under appropriate conditions, has been previously used and discussed 
in the literature [H, H]- 

We can write {SS^°^)xy = SxyS + 6Sxy, where s is the entropy of system s, 6xyS is its 
variation from y to x, and 6Sxy is the corresponding total variation of the entropy of other 
systems. For any function h of the system state x, we write Sxyh = h{x) — h{y). Thus 
SxyS = s{x) — s{y), whereas {SS)xy is the entropy variation of the external systems: it is 
supposed to depend only on x and y, as discussed below, but in general it is not the variation 
from ?/ to X of any function of the system state alone [l], 0]. If, under special conditions, 
it happens that there is a function S^°^{x) such that {6S^°^)xy = S^°^{x) — S^°^{y) for all 
the only stationary distribution p^{x) of s is p^{x) oc S^"^{x), and detailed balance 
0, i, i, S S holds 

RxyP^x) = RyxP^y); (4) 

but this is not true in general. 

We assume that the system s can exchange energy with several reservoirs S'', labelled by 
indices u = 1,2, . . .. Reservoir S'' is characterized by its temperature T^, or, better, by its 
inverse temperature jS,^ = the Boltzmann factor is always taken to be unity. The 

entropy variation of S'' is - Pu^Qu when it supplies energy (i.e., heat) Sq^, to the system. The 
system can also receive energy (i.e., work) 5w = Sq^ from a mechanical system S°, whose 
entropy, by definition, does not change in time. We characterize the mechanical system by 
its zero inverse temperature, /3o = 0. 

During an elementary transition y ^ x, we assume that the system can receive heat 6qxy 
from at most one of the reservoirs{whose inverse temperature is denoted f3xy), and work Swxy 
from the mechanical system. Thus the energy variation of the system is SxyC = 6qxy + Swxy 
We suppose, as is currently done, that during an elementary transition y ^ x the mechanical 
work 6wxy can be expressed as a function of x and y alone: then the same property holds 
for 6qxy and for the entropy variation of the reservoir which interacts with s during y ^ x. 
The total entropy variation 55**°* (7, t = Nt) along an A^-step trajectory 7 from yo to xq is 

N-l N-l 

6S^"\^,t= Nt) = [Su„u„+iS + SSu^u„+,] = Sx^y^s + ^ [-f3u„+,ujqu„+iu„] (5) 

n=0 n=0 
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Define 7 to be the time reversal of the trajectory 7. Then using relation (3) it is straight- 
forward to show that 

^ E,:„;l7l=A^,^(7) P(a^o,t|yo,0) 

Here r|?/,0) is the transitionprobability from y to x during time t. li t/r = N ^ 1, 
p{x, T\y,0) ~ p^{x). Defining [2I, the information potential of x by (f){x) = — logp'^(x) we 
obtain by (jSD 

(exp(-55*°H7, t = Nt))^^^^ = exp - ^ exp (7) 

Using the expression for 55**°* we obtain our first main result, the generalized Clausius 
relation 

N-l \ 

exp f3u„+,ujqur.+,uj ) = exp[6^,y,{s + 0)] (8) 
which can also be written 



"—0 / xoyo 



exp I Yl f^^^^'' ] I = exp[5^oy„(s + cfy] (9) 

^ft^>0 ' I xoyo 

where is the total heat received by the system from thermostat S*^ during the overall 
transition. From relation ^ we obtain by Jensen's inequality 

^^>0 / xoyo 

Multiplying flTU]) by the joint probability piyo, 0; Xq, t) and summing over xq and yo yields 

5^/3,5gM<AiS (11) 

where (■) denotes the global average over all paths between times and t, and Ajs = 
s{t) — s(0) is the variation of the macroscopic entropy s of the system between times and 
t, with 

s{t) = {s{x) - \n[p{x, t)]) = Y s{x)p{x, t) - ^p(x, t) \n[p{x, t)] (12) 

X X 

Inequalities flTUl) and flTTl) are changed into equalities if and only if in each elementary 
transition y x we have Pxy^Qxy = Sxy{s + 0) = (s + (f)){x) — {s + (p){y), which is easily 
shown to be equivalent to detailed balance, (4). Thus, detailed balance is the mesoscopic 
counterpart of the reversibility of a thermodynamic transition. 

Relation (fTTj) is the classical Clausius inequality jl] , whereas (fTOj) is a mesoscopic version 
of this inequality. They can be compared with the generalizations of the Clausius inequality 
obtained in quite different contexts by Refs. j§, [13] • If we now restrict ourselves to an 
isothermal system, (3^ = (3 for any we easily recover the result of Ref. 0] 

exp(-/3 ^ J \ =exp[5^,y„(-/3/ + 0)], (13) 

'^=0 / xoyo 
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where f{x) = e{x) — Ts{x) is the mesoscopic free energy in state x. From relation (fT3|) one 
can recover the Jarzynski equaht y which has given rise to a large literature in recent 



years (see for instance [HI, llJ, 13, 14, 15, 1^ and references therein). 

Our previous results can be extended straightforwardly to inhomogeneous systems con- 
sisting of n homogeneous cells, provided that during each elementary transition x —>■ y, each 
cell k of the system s interacts with at most one of the reservoirs, S'', whereas it receives 
work from the mechanical system and energy from the other cells. The results can also be 
extended to a system exchanging various resources with the reservoirs: this more complex 
case will be addressed in a later publication. 



As a particular consequence of the Clausius inequality, Carnot's theorem [171] gives the 
maximum efficiency in work production for a motor operating between two heat baths only, 
which is attained if all transitions are reversible. Under these conditions, however, a finite 
transition needs an infinite time and power productions vanishes. In practice, power is often 
the most relevant quantity and maximum efficiency may be less important. To address this 
problem, we now consider power and entropy production per unit time when the stochastic 
system s is in its stationary state. The probability current corresponding to the elementary 
transition y ^ x is then 

Jxy = RxyP^iy) - RyxP^ix) . (14) 

The stationary total entropy production per unit time [l| can be written, thanks to (3), 

^ x'.j/ x,y ^yxP0{X) 

Relation f|T5l) expresses the well-known fact [l|, 0] that entropy production vanishes if and 
only if the stationary state satisfies detailed balance. Defining 

^ ^ RyxPoix) - ' ^ ^ 

we have 

x,y x,y x,y x,y,u^O 

SO that V vanishes if J^y = for each transition during which s actually interacts with one 
of the reservoirs. Since V vanishes if and only if J^y = for all elementary transitions, we 
conclude that if Jxy = for each transition during which s actually interacts with a reservoir 
{Pxy 7^ 0), then = for all elementary transitions. It is found from ([3j) and (|T6!l that 

T^xy = Jxy [Sxy{s + 0) " Pxy^xyC + PxySw^y] ■ (18) 

From (ITS!) we deduce that the power received by s is 



x,y,0xy>O 



1 _ 1 7 A r 

-z i^xy " -p. JxyOxy\S + 

Pxy Pxy 



(19) 



The first term on the right hand side is always non negative. It allows one to give a general, 
explicit definition of the power dissipation 

-«-\ E/..i.ntf|!|^0. ,20) 

x,y,Pxy>0 " x,y,l3xy>0 " y \ / 
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Thus the power —V released by the system satisfies 

-V<-A=l J2 -j^JxM^ + <!>)■ (21) 

x,y,f3xy>0 ^ 

This upper bound on —V is obtained if and only if V^y = for any transition with jS^y > 0, 
which implies that Jxy = for any transition: then detailed balance is satisfied and V 
vanishes. Thus, in order that a system can act as a motor {—V > 0), a necessary condition 
is that it is not in equilibrium: the power dissipation should be positive. 

Moreover, it is seen that —A is a linear function of the currents (if the stochastic po- 
tential is supposed to be fixed), whereas the power dissipation Vy^ can be approximated 
by a quadratic function of the currents near detailed balance conditions. Under particular 
circumstances, these remarks allow us to make rough estimates of maximum power produc- 
tion and its relation to associated quantities. Because these results are quite suggestive we 
elaborate on the details. We suppose that the actual transition matrix, Rr^y is near in value 
to another, R^^y, which satisfies detailed balance. Moreover, both R and R have the same 
stationary state, p^{x). Under these circumstances a remarkable fact emerges: an upper 
bound of the power delivered by the system is obtained if the dissipation is equal to the 
power produced. Let K^y = R^yp^^y), with K the corresponding quantity for R. Then by 
assumption K^y = Ky^. To lowest order in the deviation of R from R one can easily show 
that 

Writing B^y = 5xy (s + 0), we have 



~ - / , 75 ( '^xy ~ Ti^xyBxy 1 + -rKxy {Bx 



2 

y. 



< 



I E ^iBxyf = -V--. (23) 



This upper bound is obtained if Jxy = \KxyBxy for each transition, in which case the power 
dissipation is equal to the power produced: 

pmax^l ^ ^^Bxyf = -V^^''. (24) 

In this situation, the power produced is half the quantity —A = —V + V, given by Eqs. (fT9|) 
and (12T]) . It is clear that the currents must satisfy constraints which may not allow reaching 
this optimal situation. Nevertheless, maximizing —V under the relevant constraints confirms 
that close to detailed balance, the maximum power released by the system is obtained when 
the power dissipation is of the same order of magnitude as the power produced. 

Of course, this may be invalid far from detailed balance conditions. More accurate, 
quantitative results should rely on specific examples, but in principle they can be obtained 
from the previous general tools. In this connection we mention a tantalizing example [ll] 
from an elementary mechanics text. It is an exercise: "Material drops from a hopper at a 
constant rate dm/dt onto a conveyor belt moving with constant velocity v parallel to the 
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ground. What power motor would be needed to drive the belt?" The answer turns out to 
be that the optimum power to be supplied is exactly twice the kinetic energy imparted to 
the particles (which can be seen by going into the belt reference frame). Thus the power 
output (the kinetic energy of the material) exactly equals the energy dissipated by friction. 
Our assumptions in the foregoing derivation are too restrictive to make this result a special 
case, but there is very much the suggestion that the factor 1/2 that we have encountered is 
more general than our demonstration. 
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